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Abstract — We address the problem of securing distributed 
storage systems against passive eavesdroppers that can observe 
a limited number of storage nodes. An important aspect of 
these systems is node failures over time, which demand a repair 
mechanism aimed at maintaining a targeted high level of system 
reliability. If an eavesdropper observes a node that is added to the 
system to replace a failed node, it will have access to all the data 
downloaded during repair, which can potentially compromise the 
entire information in the system. We are interested in determining 
the secrecy capacity of distributed storage systems under repair 
dynamics, i.e., the maximum amount of data that can be securely 
stored and made available to a legitimate user without revealing 
any information to any eavesdropper. We derive a general upper 
bound on the secrecy capacity and show that this bound is 
tight for the bandwidth-limited regime which is of importance 
in scenarios such as peer-to-peer distributed storage systems. We 
also provide a simple explicit code construction that achieves the 
capacity for this regime. 



I. Introduction 



Data storage devices have evolved significantly since the 
days of punched cards. Nevertheless, storage devices, such as 
hard disks or flash drives, are still bound to fail after long 
periods of usage, risking the loss of valuable data. To solve 
this problem and to increase the reliability of the stored data, 
multiple storage nodes can be networked together to redun- 
dantly store the data, thus forming a distributed data storage 
system. Applications of such systems are innumerable and 
include large data centers and peer-to-peer storage systems, 
such as OceanStore JT], that use a large number of nodes 
spread widely across the Internet to store files. 

Codes for protecting data from erasures have been well 
studied in classical channel coding theory, and can be used 
here to increase the reliability of distributed storage systems. 
Fig. Q] illustrates an example where a maximal distance sep- 
arable (MDS) code is used to store a file J 7 of 4 symbols, 
(ai, a2, bi, 62) S F|, distributively on 4 nodes, vi,...,Va, 
each of capacity 2 symbols. The MDS code implemented here 
ensures that any user, also called data collector, connecting to 
any 2 storage nodes can obtain the whole file T . However, 
what distinguishes the scenario here from the erasure channel 
counterpart is that when a storage node fails, it needs to be 
repaired or replaced by a new node in order to maintain a 
desired level of system reliability. A straightforward repair 
mechanism would be to add a new replacement node of 
capacity 2, and make it act as a data collector by connecting 
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Fig. 1. An example of a distributed data storage system under repair. A 
file J 7 of 4 symbols (ai, 0,%, 61, 62) £ is stored on four nodes using an 
MDS code. Node v\ fails and is replaced by a new node v$ that downloads 
(bi + b 2 ), (ai + a 2 + 61 + b 2 ) and (ai + 4a 2 + 2bj + 2b 2 ) from v 2 , 
V3, and «4 respectively to compute and store (ai + a 2 , ai + 4a 2 ). Nodes 
v 2 , . . . , vs form a new MDS code. The edges in the graph are labeled by 
their capacities. The figure also depicts a data collector connecting to nodes 
i>2 and V4 to recover the stored file. 



to 2 surviving nodes. The new node can then download the 
whole file (4 symbols) to construct the lost part of the data and 
store it. Another repair scheme that consumes less bandwidth 
is depicted in Fig. Q] where node V\ fails and is replaced by 
node V5. When node v$ connect to 3 nodes instead of 2, it 
is possible to decrease the total repair bandwidth from 4 to 3 
symbols. Note that v$ does not need to store the exact data that 
was on i>i; the only required property is that the data stored 
on all the active nodes 1)2,1)3,1)4 and w 5 form an MDS code. 

The above important observations were the basis of the 
original work of J2) where the authors showed that there 
exists a fundamental tradeoff between the storage capacity of 
each node and the repair bandwidth. They also introduced and 
constructed "regenerating codes" as a new class of codes that 
generalize classical erasure codes and permit the operation 
of a distributed storage system at any point on the tradeoff 
curve. When a distributed data storage system is formed using 
nodes widely spread across the Internet, e.g., Internet based 
peer-to-peer systems, individual nodes may not be secure and 
can become susceptible to eavesdropping. This paper focuses 
on such scenarios where an eavesdropper can gain access 
to a certain number of the storage nodes. The compromised 
distributed storage system is always assumed to be dynamic 
with nodes continually failing and being repaired. Thus, the 
compromised nodes can belong to the original set of storage 



nodes that the system starts with, or even include some of 
the replacement nodes added to the system to repair it from 
failures. Under this setting, we are interested in determining 
how much data can still be stored in the system without 
revealing any information to any of the eavesdroppers. 

To answer this question, we follow the approach of (2) and 
model the distributed storage system as a multicast network 
that uses network coding. Under this model, the eavesdropper 
is an intruder that can access a fixed number of the network 
nodes of her choice. This eavesdropper model is natural for 
distributed storage systems and comes in contrast with the 
wiretapper model studied in the network coding literature 0, 
ID, |5) where the intruder can observe network edges, instead 
of nodes. We derive a general upper bound on the secrecy ca- 
pacity as a function of the node storage capacity and the repair 
bandwidth. Motivated by system considerations, we define an 
important operating regime, that we call the bandwidth-limited 
regime, where the repair bandwidth is constrained not to 
exceed a given upper bound, while no limitation is imposed on 
the storage capacity of the nodes. For this important operating 
regime, we show that our upper bound is tight and present 
capacity-achieving codes. 

This paper is organized as follows. In Section[n]we describe 
the system and security model. We define the problem and 
give a summary of our results in Section [ill] In Section |IV] 
we illustrate two special cases of distributed storage systems 
that are instructive in understanding the general problem. In 
Section [V] we derive an upper bound on the secrecy capacity, 
and in Section [VT] we present a scheme that achieves this 
upper bound for the case of bandwidth-limited regime. We 
conclude in Section \VU\ 

II. Model 

A. Distributed storage system 

A distributed storage system (DSS) is a collection of storage 
nodes that includes a source node s, that has an incompressible 
data file T of R symbols, or units, each belonging to a finite 
field F. The source node is connected to n storage nodes 
V\, . . . , v ni each with a storage capacity of a symbols, which 
may be utilized to save coded parts of the file F. The storage 
nodes are individually unreliable and may fail over time. To 
guarantee a certain desired level of reliability, we assume that 
the DSS is required to always have n active, i.e., non-failed, 
storage nodes that are in service. Therefore, when a storage 
node fails, it is immediately replaced by a new node with same 
storage capacity a. The DSS should be designed in a way to 
allow any legitimate user, that we also call data collector, that 
connects to any k out of the n active storage nodes available 
at any given time, to be able to reconstruct the original file 
T . We term this condition as the "reconstruction property" of 
distributed storage systems. 

We assume that nodes fail one at a time, and we denote by 
v n+ i the new replacement node added to the system to repair 
the i-th failure. The new replacement node connects then to 
some d nodes, chosen randomly, out of the remaining active 
n — 1 nodes and downloads 7 units from them in total, which 
corresponds to the repair bandwidth of the system. The repair 
degree d is a system parameter satisfying k < d < n—1. In this 



work, we focus on the case of symmetrical repair where the 
new node downloads equal amount of data, say j3 units, from 
each of the d nodes it connects to, i.e., 7 = dj3. The process of 
replenishing redundancy to maintain the reliability of a DSS 
is referred to as the "regeneration" or "repair" process. Note 
that a new replacement node may download more data than 
what it actually stores. Moreover, the stored data can possibly 
be different than the one that was stored on the failed node, as 
long as the "reconstruction property" of the DSS is retained. A 
distributed storage system V is thus characterized as T>(n, k). 
For instance, the DSS depicted in Fig.[T]corresponds to T>(4, 2) 
which is operating at (a, 7) = (2,3). 

B. Flow Graph Representation 

We adopt the flow graph model introduced in |2 | which we 
describe here for completeness. In this model, the distributed 
storage system is represented by an information flow graph 
Q. The graph Q is a directed acyclic graph with capacity 
constrained edges that consists of three kinds of nodes: a single 
source node s, input storage nodes x\ n and output storage 
nodes x l out and data collectors DC, for i,j <G {1,2,...}. 
The source node s has an information S of which a specific 
realization is the file F. Each storage node in the DSS is 
represented by two nodes x\ n and x l out joined by a directed 
edge of capacity a (see Fig.©, to account for the node storage 
constraint. 

The repair process is initiated every time a failure occurs. 
As a result, the DSS, and consequently the flow graph, are 
dynamic and evolve with time. At any given time, each node 
in the graph is either active or inactive depending on whether 
it has failed or not. The graph Q starts with only the source 
node s being active and connected to the storage input nodes 
x\ n , . . . , x™ n by outgoing edges of infinite capacity. From 
this point onwards, the source node s becomes and remains 
inactive and the n input and output storage nodes become 
active. When a node Vi fails in a DSS, the corresponding nodes 
x\ n and x l out become inactive in Q. If a replacement node Vj 
joins the DSS in the process of repairing a failure and connects 
to d active nodes , . . . , Vi d , the corresponding nodes x\ n and 
x J out , with the edge {x J in ,x J out ), are added to the flow graph 
Q, and node x\ n is connected to the nodes x* „ t , . . . , x l d ut 
by incoming edges of capacity /3 each. A data collector is 
represented by a node connected to k active storage output 
nodes through infinite capacity links enabling it to reconstruct 
the file T. The graph Q constitutes a multicast network with 
the data collectors as destinations. An underlying assumption 
here is that the flow graph corresponding to a distributed 
storage system depends on the sequence of failed nodes. As 
an example, we depict in Fig. [2] the flow graph corresponding 
to the DSS D(4, 2) of Fig. Q] when node v Y fails. 

C. Eavesdropper Model 

We assume the presence of an intruder "Eve" in the DSS, 
who can observe up to I, I < k, nodes of her choice among 
all the storage nodes, V\ , V2 , • • • , possibly at different time 
instances as the system evolves. In the flow graph model, Eve 
is an eavesdropper who can access a fixed number £ of nodes 
chosen from the storage input nodes x\ n , xf n , . . . . Notice that 
while a data collector observes output storage nodes, i.e., the 




Fig. 2. The flow graph model of the DSS X>(4, 2), with d = 3, of Fig. Q] 
when node fails and is replaced by node tig. Each storage node is 
represented by two nodes x\ n and xj, ut connected by an edge {x z in , x' out ) 
of capacity a representing the node storage constraint. A data collector DC 
connecting to nodes vi and V4 is also depicted. 

data stored on the nodes it connects to, Eve, has access to 
input storage nodes, and thus can observe, in addition to the 
stored data, all incoming messages to these nodes. We also 
assume that Eve has complete knowledge of the storage and 
repair schemes implemented in the DSS. Thus, she can choose 
some of the i nodes to be among the initial n storage nodes, 
or, if she deems it more profitable, she can choose to wait for 
failures and eavesdrop on a replacement node by observing 
its downloaded data. Eve is assumed to be passive, and only 
observes the data without modifying it. 

III. Problem Statement and Results 
A. Secrecy Capacity 

Let S be a random vector uniformly distributed over F^, 
representing the incompressible data file at the source node 
with H(S) = R. Let V in := {x\ n , xf n , . . . } and V ou t ■= 
{xl ut , x 2 out , . . . } be the sets of input and output storage nodes 
in Q respectively. For a storage node vi, let Di and Cj be 
the random variables representing its downloaded messages 
and stored content respectively. Thus, Cj, represents the data 
that can be downloaded by a data collector when contacting 
node Vi, while Di, with H(Di) < 7, represents the total data 
revealed to Eve when she accesses node Vj. The stored data 
Ci is a function of the downloaded data Di. 

Let Vg Ut be the collection of all subsets of V ou t of car- 
dinality k consisting of nodes that are simultaneously active 
at some instant in time. For any subset B of V ou t, define 
c B ■■= {a : Xq u4 £ B}. Similarly, for any subset E of Vj n , 
define De ■= {Di : x\ n £ E}. The reconstruction property, 
then, can be written as 

H(S\C B ) = VB6C, (1) 
and the perfect secrecy condition implies 

H{S\D E ) = H(S),VE C V in and \E\ < £. (2) 

Given a DSS T>(n, k) with I compromised nodes, its secrecy 
capacity, denoted by C s (a, 7), is then defined to be the 
maximum amount of data that can be stored in this system 
such that the reconstruction property and the perfect secrecy 



condition are simultaneously satisfied for all possible data 
collectors and eavesdroppers i.e., 

C s (a, 7 ):= sup H(S) (3) 

H(S\C B ) = VB 
H(S\D E ) = H(S) VE 

where B £ V^ ut , E C V in and \E\ < i. 

B. Results 

First, we give the following general upper bound on the 
secrecy capacity of a DSS: 

Theorem 1: [Upper Bound] For a distributed data storage 
system T>(n, k), with a repair degree d, and I < k compro- 
mised nodes, the secrecy capacity is upper bounded as 

k 

G s {a^) < J2 min{(d-i + l)j8,a}, (4) 

i=(+l 

where 7 = 

Next, we consider an important operational regime, namely 
the bandwidth-limited regime, where the repair bandwidth 7 
is constrained to a maximum amount T, i.e., 7 < T, while no 
constraint is imposed on the storage capacity a at each node. 
The secrecy capacity in this regime is defined as, 

Cf L {T):= sup C>, 7 ). (5) 
7 < T,0 < a 

For a fixed T, when the parameter ci is a system design choice, 
the upper bound of Theorem [T] on the secrecy capacity can be 
further optimized, and attains a maximum for d = n — 1. 
In section I VII we demonstrate that this upper bound can be 
achieved for d = n — 1 in the bandwidth-limited regime. Thus, 
establishing the following theorem: 

Theorem 2: [Bandwidth-Limited Regime] For a distributed 
data storage system T)(n, k), £ < k compromised nodes, the 
secrecy capacity for a bandwidth-limited regime, for d = n— 1, 
is k 

Cf L {T)= ]T („_»)_£_, (6) 

i=£+l 

and is achieved with a storage capacity of a = V. 

IV. Special Cases 

A. Static Systems 

A static version of the problem studied here corresponds 
to a DSS with ideal storage nodes that do not fail. Hence 
there is no need for any repair in the system. The flow graph 
of this system is then the combination network studied in 
network coding theory (see for e.g. J6] Chap. 4] ). Therefore, 
the static storage problem can be regarded as a special case 
of wiretap networks J3), Q, or equivalently, as the erasure- 
erasure wiretap-II channel studied in J7). The secrecy capacity 
for such systems is (k — £)a, and can be achieved using either 
nested MDS codes Q, or the coset codes of (8), 0. 

Even though the above proposed solution is optimal for the 
static case, it can have a very poor secrecy performance when 
applied directly to dynamic storage systems with failures. For 
instance, a straightforward way to repair a failed node would 
be to download the whole file on the new replacement node, 




Fig. 3. Part of the flow graph corresponding to a DSS T>(n, k), when nodes 
Vi,...,V£ fail successively, and are replaced by nodes v n +i, . . . , u n -|-fc. A 
data collector DC connects to these k nodes and wants to retrieve the whole 
file. Nodes i> n +i, • ■ • , i>n+i shown with broken boundaries are compromised 
by Eve during repair. 

and then generate the specific lost data. In this case, if Eve 
accesses the new replacement node while it is downloading 
the whole file, it will be able to reconstruct the entire original 
data. Hence, the secrecy rate for this scheme would be zero. 
However, Theorem [2] suggests that for some systems we can 
achieve a positive secrecy capacity. This example highlights 
the fact that dynamical repair of the DSS renders it intrinsically 
different from the static counterpart, and one should be careful 
in designing the repair scheme in order to safeguard the whole 
stored data. 

B. Systems Using Random Network Coding 

Using the flow graph model, the authors of |2] showed 
that random linear network codes over a large finite field 
can achieve any point (a, 7), on the optimal storage-repair 
bandwidth tradeoff curve with a high probability. Consider 
an example of random linear network code used in a com- 
promised DSS X>(4, 3), which stores R = 6 symbols and 
operates at d = 3,/? = 1, and a = 3. In this case, each of 
the initial nodes v±, . . . , 1)4 stores 3 independently generated 
random linear combinations of these R = 6 symbols. Assume 
now that node V4 fails and is replaced by a new node V5 that 
connects to Vi,v 2 , and i> 3 , and downloads from each one of 
them j3 = 1 random linear combination of their stored data. 
Assume that after some time, node W5 fails and is replaced by 
node vq in a similar fashion. Now, if £ — 2, and Eve accesses 
nodes v$ and vq while they were being repaired, it will observe 
6 linear combinations of the original data symbols, which, with 
high probability are linearly independent. Therefore, she will 
be able to reconstruct the whole file. 

The above analysis shows that, when random network 
coding is used, it is not possible to achieve a positive secrecy 
rate for this system, even with pre-processing at the source, 
using for example Maximum Rank Distance (MRD) codes |5j. 
But according to Theorem [2] which we prove in section [VlJ 
the secrecy capacity of the the above DSS D(4, 3) is equal to 
one unit when 1 = 2. This is also in contrast with the case of 
multicast networks with compromised edges instead of nodes 
J3|, wherein, random network coding can perform as good as 



any deterministic secure code Q. 

V. Upper bound on secrecy capacity 

In this section we derive the upper bound of Theorem [TJ 
Consider a DSS T>(n, k) with £ < k. Assume that the nodes 
v\ , V2, ■ ■ ■ , Vk have failed consecutively, and were replaced 
during the repair process by the nodes u n +i, tVt+2, • • • , Vn+k 
respectively as shown in Fig. [3] Now suppose that Eve accesses 
nodes in E = {tVt+i, u n +2, ■ • ■ , Vn+e} while they were being 
repaired, and consider a data collector connected to the nodes 
in B = {v n+ i, v n +2, ■ ■ ■ , v n +k}- The reconstruction property 
implies H(S\Cb) — by Eq. ([TJ, and the perfect secrecy 
condition implies H{S\D E ) = H(S) by Eq. ©. We can 
therefore write 

H(S) = H(S\D E ) - H(S\C B ) 

< H(S\Ce)-H(S\C b ) 

= H(S\Ce) — H(S\Ce,C b \ e ) 

= I(S, C b \ e \Ce) 

<H(C b \ e \Ce) 
k 

= H(C n +i\C n .\-i, . . . ,C n +i-i) 
i=e+i 

(3) *\ 

< ^2 min{(d - i + a}. 

i=t+i 

Inequality (1) follows from the fact that the stored data Ce 
is a function of the downloaded data D B , (2) from, C B \ B '■= 
{C n +e+i, • ■ • , Cn+fc}, (3) follows from the fact that each node 
can store at most a units, and for each replacement node we 
have H(Ci) < H{Di) < dp, also from the topology of the 
network (see Fig. |3j. Note that each node x"^ 1 is connected 
to each of the nodes x™£t , . . . , a;"^ 1-1 by an edge of capacity 
f3. The upper bound of Theorem [TJ follows then directly from 
the definition of Eq. d3). 

VI. Secrecy Capacity in the Bandwidth-Limited 
Regime 

A. Example 

Consider again the DS S £>(4 , 3) with a = 3, d = 3, B = 1, 
and £ = 2 of Section IIV-BI for which the secrecy rate 
using random linear network coding was shown to be 0. The 
upper bound on the secrecy capacity of this system given 
by Theorem [TJ is 1, We provide a scheme that achieves 
this upper bound. The proposed code is depicted in Fig. 0] 
and consists of the concatenation of an MDS coset code 
[8 1 with a special repetition code that was introduced in (9) 
by Rashmi et al. for constructing exact regeneration codes. 
Let S G F q denote the information symbol to be securely 
stored on the system. S is encoded using the outer MDS code 
into a codeword (Z, Ki,K 2 , ■ ■ ■ , K$), where Ki, . . . , K$ are 
independent random keys uniformly distributed over ¥ q and 
Z = S + ^2 i=1 Ki. The encoded symbols Z, K\ , . . . , K§ 
are then stored on the nodes v%, . . . , U4 as shown in Fig. @] 
following the special repetition code of @. It is easy to 
verify that any data collector connecting to 3 nodes, observes 
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Fig. 4. Schematic representation of the optimal code for the DSS TJ(A, 3) 
with a = 3, (3 = I, d = 3, and 1 = 2 that achieves the secrecy capacity of 1 
unit. An MDS coset code takes the information symbol S and five independent 
random keys K\, . . . , K$, as an input and outputs a parity check symbol 
Z = S + A'i, along with random keys in systematic form. These 

symbols are then stored on the DSS using the code structure of (9). 

all the symbols Z, K\, . . , , K$, and can therefore decode 
S = Z — J2i=i Ki- However, an eavesdropper accessing 
any two nodes will only observe 5 symbols out of 6, and 
cannot gain any information about S. Next, we generalize 
this construction to obtain a capacity-achieving code for the 
bandwidth-limited regime. 

B. Code Construction 

Our approach builds on the results of (9) where the authors 
constructed a family of exact regenerating codes for d = n—1. 
The "exact" property of these codes allows any repair node to 
reconstruct and store an identical copy of the data lost upon 
a failure. For simplicity, we will explain the construction for 
/3 = 1, i.e., r = For any larger values of T, and in turn of 
(3, the file can be split into chunks, each of which can be sepa- 
rately encoded using the construction corresponding to (3 = 1. 
Choose a = T. From J2] we know that M = Yli=i( n — i) is 
the capacity of the above DSS in the absence of any adversary 
(I = 0). Let R := X)i=f+i ( n ~ *) ^ e me number of information 
symbols that we would like to store securely on the DSS, 
and 8 := ^=^-. Let S = (s u ...,s R ) € Ff denote the 
information file and K = (K 1: . . . , K M -r) S F^ /_i? denote 
M — R independent random keys each uniformly distributed 
over Wg, Then, the proposed code consists of an outer nested 
(9, M) MDS coset code Q which takes S and /C as an input, 
and outputs X = [x\, . . . , xq), such that X = JCGk + SGs, 

where G — ( J is a generator matrix of a (8, M) MDS 

code, and Gk in itself is a generator matrix for a (9,M — R) 
MDS code. The information vector S effectively selects the 
coset of the MDS code generated by Gk- 

This outer (6, A/) MDS code is then followed by the special 
repetition code introduced in [9] which stores the codeword 
X on the DSS. The procedure of constructing this inner code 
can be described using an auxiliary complete graph over n 
vertices m, ■ ■ ■ , u n that consists of 8 edges. Suppose the edges 
are indexed by the coded symbols x±, . . . , xg. The code then 
consists of storing on node Vi the indices of the edges adjacent 
to vertex Ui in the complete graph. Consequently, every coded 
symbol Xi is stored on exactly two storage nodes, and any pair 
of two storage nodes have exactly one distinct coded symbol 
in common, e.g., code in Fig. [4] for n = 4. 

This inner code transforms the dynamic storage system into 



an equivalent static point-to-point channel. First notice that 
a = r, hence all the data downloaded during the repair 
process, i.e., d/3 = T, is stored on the new replacement 
node without any further compression. Thus, accessing a node 
during repair process, i.e., observing its downloaded data, 
is equivalent to accessing it after the repair process, i.e., 
observing its stored data. Second, the exact regeneration codes 
restore a failed node with the exact lost data. So, even though 
there are failures and repairs, the data storage system looks 
exactly the same at any point of time. Any data collector 
downloads M symbols out of xi,...,x$ by connecting to 
k nodes. Moreover, any eavesdropper can observe /i = 
Yli—i( n — i) = M — R symbols. Thus, the system becomes 
similar to the erasure-erasure wiretap channel-II of parameters 
(9, M, fj,j^\. Therefore, since the outer code is a nested MDS 
code, from |7j| we know that it can achieve the secrecy capacity 

of M - (i = M - (M - R) = R = Ei=i+i( n - i) of 
the corresponding erasure-erasure wiretap channel. This rate 
is achieved for every 1 unit of /3. Thus, the total secrecy rate 
achieved for /3 = T/(n - 1) is Y,^=e+i( n ~ *)^"T- 

VII. Conclusion 

In this paper we considered dynamic distributed data storage 
systems that are subject to eavesdropping. Our main objective 
was to determine the secrecy capacity of such systems, i.e., 
the maximum amount of data that these systems can store and 
deliver to data collectors, without revealing any information 
to the eavesdropper. Modeling such systems as multicast 
networks with compromised nodes, we gave an upper bound 
on the secrecy capacity, and showed that it can be achieved 
in the important bandwidth-limited regime where the nodes 
have sufficient storage capacity. Finding the general expression 
of the secrecy capacity of distributed storage systems, and 
more generally of multicast networks with a fixed number of 
compromised nodes, remains an open problem that we hope 
to address in future work. 
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